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ePortfolios have been looked to as a tool for the direct assessment of student learning. Because the 
evidence used for assessment is actual student work, ePortfolios provide a view of learning that is 
not available through traditional methodologies such as student surveys and exams. This research 
examined 47 student ePortfolios completed as part of a first-year seminar course. Learning outcomes 
were assessed using both a rubric and identification of authentic evidence in the form of words and 
phrases to support rubric scores. Findings indicated that the ePortfolio is a rich source of evidence 
from which to assess learning and the authentic evidence was closely aligned with rubric scores. 

Rubric scores indicated the level of learning that occurred while student narrative provided insight 
into the level of student thinking and depth of thought around particular topics including 
understanding of self and exploration of majors and careers. When coupled together, an analytic 
rubric and examination of student narrative as authentic evidence provided a robust methodology for 
assessing student learning. 


As institutions across the United States seek to find 
ways to increase graduation and deepen learning, 
educational strategies including intrusive advising and 
mentoring, engaging classroom pedagogies, and high- 
impact practices are increasingly being implemented on 
campuses and in classrooms. Many campuses have 
focused on first-year students in an effort to build 
foundations for critical thinking, engagement on 
campus, and commitment to college completion. Others 
have used ePortfolios as a tool for assisting students in 
deepening and documenting learning in a course, 
academic program, or across the college experience. 
Regardless of the strategies implemented, most 
institutions have measured the success of initiatives 
using quantitative data such as grade point average, 
retention rates, and graduation numbers. 

This paper presents research completed to explore 
the outcomes of two educational strategies, first-year 
seminars and ePortfolios, in a way that goes beyond 
traditional quantitative measures of success. Using 
student narrative in an ePortfolio as authentic evidence 
of student learning, researchers sought to determine the 
level of achievement for five learning outcomes 
associated with a first-year seminar course. 

Literature Review 

ePortfolios 

The electronic portfolio (ePortfolio) has emerged 
over the last decade as one tool for responding to the 
pressures facing higher education in the areas of 
assessment and accountability for student learning 
outcomes (Cambridge, 2001; Chen & Penny Light, 
2010; Watson & Doolittle, 2011). Banta (2003) posited 
that “portfolios enable faculty to see firsthand not only 


what students are learning, but how they are learning” 
(p. 2) in addition to the observation that “portfolios also 
can play a role in assessing the effectiveness of the 
courses, curricula, and even institutions” (p. 4). The 
ePortfolio is also used to support and document the 
personal, professional, and intellectual development of 
students (Watson & Doolittle, 2011). 

Zubizarreta (2004) referred to portfolios that 
support student’s affective and cognitive development 
as learning portfolios. Learning portfolios can be used 
for a variety of purposes across a range of settings 
including the classroom, co-curricular programs, and at 
the program or institutional level. Across purposes and 
settings, Zubizarreta (2004) suggested all learning 
portfolios have three primary components: (a) 
documentation as evidence and outcomes of learning, 
(b) reflections on learning, and (c) collaboration and 
mentoring most often in the form of faculty feedback. 
Frequently, learning portfolios are structured to serve as 
living documents that emphasize the learning that 
occurs through the process of developing a portfolio, 
rather than the portfolio itself as an outcome (Seimens, 
2006, as cited in Garris, 2007). Because the focus of 
learning portfolios is on the individual’s orientation and 
process of learning, they often are considered to be 
personal or developmental portfolios. 

While some see the use of portfolios for assessment 
and learning and personal portfolios focused on learning 
process and student development as two distinct types of 
portfolios, Cambridge (2010) posited that they both 
contribute to the ideal of authenticity where authenticity 
is undergirded by, "the principle that we do not really 
understand our unique selves or participate fully in life 
until we express our natures” (p. 13). Portfolios designed 
for assessment purposes allow for measurement of 
student learning against an established standard while 
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personal portfolios allow the learner to define and narrate 
their learning as they see it. Both provide significant 
information on learning to the institution or program as 
well as the student. Therefore, the potential for 
ePortfolios as tools for both assessment and support for 
student learning is tremendous and this potential can be 
maximized if the portfolio is developed beginning in the 
first year of college. 

First-Year Seminars 

It has long been recognized that the first year of 
college is a significant point in time for both the student 
and the institution (Barefoot et al., 2005). From the 
student perspective, this is a year of significant 
challenge and change. Students are faced with having to 
engage in independent decision-making, more rigorous 
classroom expectations, interacting with diverse people 
and perspectives, and a maturing sense of identity. 
From the institutional perspective, assisting the student 
in developing the knowledge, understanding, and skills 
that promote college success reduces the large rates of 
attrition seen between the first and second year of 
enrollment. A wide array of institutional interventions 
to support entering students have been employed at 
campuses across the United States including orientation 
programs, learning communities, academic advising, 
supplemental instruction, and first-year seminars 
(Upcraft, Gardner, & Barefoot, 2005). 

First-year seminars are as diverse as the institutions 
offering them. They may be academically-focused and tied 
to an intellectual theme, focused on basic study skills, or 
serve as an extended orientation to the campus—or a 
combination of all three (Griffin & Romm, 2008). The 
seminars may serve beginning students in their first 
semester or through the first-year of enrollment, be offered 
for credit or not, and be graded or marked pass-fail. 
Similarly, assessment methodologies for first-year 
seminars are as varied as their purposes, ranging from 
analysis of GPA and retention to measures of critical 
thinking and civic engagement. 

Kuh (2008) argued that first-year seminars and 
experiences are one of 10 high-impact practices that 
increase student engagement and learning. The most 
successful seminars focus on engagement of students 
through critical inquiry, frequent writing, 
information literacy, and collaborative learning. Kuh 
and O’Donnell (2013) went on to indicate that in 
order to be considered “high-impact” in their efforts 
to foster outcomes such as improved retention, on- 
time graduation, and deeper learning, practices such 
as first-year seminars must meet eight conditions: (1) 
performance expectations must be set at 
appropriately high levels; (2) students must invest 
significant time and effort over an extended period of 
time; (3) students must interact with faculty and 
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peers about substantive matters; (4) students should 
have experiences with diversity; (5) instructors 
should provide students with frequent, timely, and 
constructive feedback; (6) instructors should provide 
students with periodic, structured opportunities to 
reflect and integrate learning; (7) students should 
have opportunities to discover the relevance of 
learning through real-world applications; and (8) 
students should publicly demonstrate their 
competence. These conditions are also foundational 
principles for the use of ePortfolios. In a first-year 
seminar, over the course of the first semester or year 
of enrollment, the use of a personal or learning 
portfolio can be a pedagogical technique to engage 
students in using reflection to think critically about 
themselves and apply their learning to their college 
experience while receiving feedback from a faculty 
member. Thus, ePortfolios, like the first-year 
seminar, are quite likely a high-impact practice. 

ePortfolio as Authentic Assessment 

The electronic portfolio has become increasingly 
attractive to faculty who seek a more comprehensive 
insight into and interactive approach with respect to the 
authentic assessment of their students’ process of 
learning and development (Banta, 2003). The notion of 
authenticity in assessment is based on the idea that a 
more representative evaluation of a student’s learning is 
based on evidence that represents a reflective, 
intentional timespan rather than arbitrary points in time. 
According to Cambridge (2010), 

Putting the ideals of authenticity and deliberation 
into action, ePortfolios offer one means of 
generating a comprehensive account of students’ 
experience of ineffable outcomes, having the 
promise to capture the complexity and context of 
students’ learning in ways that more conventional 
kinds of assessment cannot, (p. 118) 

The ePortfolio is a natural fit for a high impact 
practice such as a first-year seminar because of the 
integrative learning opportunities fostered by this 
structured reflection and assessment framework. 
Established learning outcomes can be assessed by 
either formative or summative means through a wide 
range of authentic evidence documented by a student 
over time (Banta, Griffin, Flateby, & Kahn, 2009). 
“As ‘containers’ of authentic evidence of student 
work, e-portfolios can serve as a catalyst for 
conversations among faculty and other stakeholders 
within departments and programs about common 
learning outcomes, coherence among courses, and 
professional development” (Chen & Penny Light, 
2010, p. 3). 
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Bringing it All Together 

At Indiana University-Purdue University 
Indianapolis (IUPUI), all first-time full-time students 
are required to enroll in a first-year seminar course 
during their initial semester of enrollment. The course 
aims to support students in building a solid foundation 
for college success. Students are introduced to key 
information and skills needed to succeed at IUPUI as 
they explore the academic culture of the campus. One 
of the foundational goals of the course is to have 
students complete an electronic personal development 
plan (IUPUI, 2010). 

The electronic personal development plan (ePDP) is 
a process that enables students to understand, implement, 
and chart progress toward their degree and college goals 
and, in doing so, become empowered to take charge of 
their own education. Specifically, the ePDP is a personal, 
developmental ePortfolio designed to foster goal 
commitment, academic achievement, curricular 
coherence, meaning-making, and student development 
(IUPUI, 2012). Students complete guided reflection 
prompts across seven distinct sections: About Me, 
Educational Goals and Plans, Career Goals, Academic 
Showcase, My College Achievements, and Resume. The 
guided prompts are aligned with stated learning 
outcomes and are evaluated using rubrics based on 
elements of critical thinking and cognitive development. 
Faculty are free to integrate as many or as few of the 
sections into their course as they see fit. Through 
professional development workshops, faculty are 
encouraged to purposefully scaffold each section, as well 
as the entire portfolio, into the course. As a result, 
students enrolled in a first-year seminar course in which 
the faculty member has chosen to use the ePDP as a tool 
for learning and development leave the course with a 
foundational portfolio that provides guidance throughout 
their college experience and serves as a repository for 
evidence and reflections on their learning. 

Research Aims 

The purpose of this research project was to use 
the student narrative found in the ePDP as a source of 
authentic evidence to evaluate the stated outcomes of 
a first-year seminar. Specifically, this project sought 
to evaluate evidence of the stated learning outcomes 
for the ePDP across the portfolio as a whole—rather 
than within each individual section of the portfolio, 
which is the current practice associated with grading 
the ePDP as a class assignment. The articulated 
learning outcomes for the ePDP as utilized in first- 
year seminars are: 

• Self-Assessment and Awareness: students will 
identify success-related competencies. 
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• Exploration: students research and identify 
realistic and informed academic and career 
goals. 

• Goal Setting: students set short and long term 
goals as well as connect personal values and 
life purpose to the motivation and integration 
behind their goals. 

• Planning: students locate programs, information, 
people, and opportunities to support and their 
goals and engage in reality checks. 

• Evaluation: students analyze their academic 
programs in terms of progress toward 
academic and career goals. 

Method 

According to Hansen and Borden (2006), “action 
research facilitates the connection between evaluation 
research results and program improvement” (p. 49). 
This project is a type of action research, as it sought to 
evaluate student learning outcomes in order to facilitate 
improvement in both the first-year seminar and the use 
of an ePortfolio as a tool for supporting learning within 
the course. Because the principal investigator in this 
project is also the project coordinator for the ePDP, the 
project supports Craig’s (2009) assertion that action 
research is conducted by a practitioner studying an 
existing issue for the purpose of improvement. 

Qualitative research is best used when the research 
is focused on “process, meaning, and understanding in 
words and pictures” (Creswell, 1994, p. 145). Since this 
project is focused on finding evidence of learning in 
student narrative, qualitative research methods were 
employed. This research orientation allows for the 
development of thick description that can describe and 
explain the data in a way that allows for a holistic 
perspective and understanding. Further, qualitative 
methods allow for inductive analysis in which the 
findings emerge from the raw data—in this case— 
student narrative. 

Selection of Portfolios 

This study was conducted at IUPUI in the fall 2012 
semester. Beginning freshmen student enrollment was 
2,811, of which 2,430 (86.4%) were enrolled in a first- 
year seminar course during their first semester of 
enrollment. Of the 121 sections of the seminar being 
taught that term, 45 of these sections used the electronic 
ePDP in the course for a total of 898 students 
completing an ePDP. Informed consent forms were 
distributed to each class for students to provide 
permission for their ePDP to be used in institutional 
research; 397 students provided this consent. From the 
population of portfolios for which consent was 
provided, an initial attempt was made to randomly 
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select a representative sample of 50 portfolios. 
However, due to security problems that impeded access 
to individual ePDPs in the portfolio platform by the 
researchers, the project used convenience sampling. 
The 397 student portfolios were sorted by class 
section and then randomized to ensure distribution 
across first-year seminars. An attempt to access each 
portfolio was made with 12% being accessible to 
researchers in the ePortfolio platform. This sampling 
technique resulted in a total of 47 portfolios analyzed 
for this study. These portfolios were implemented in 
15 different class sections of the first-year seminar, 
taught by 13 different faculty members. Seventy-eight 
percent of the portfolios were completed by female 
students. 

Development of the Rubric 

The generally accepted definition of a rubric states 
that it is a tool used in scoring qualitative student work 
that includes both dimensions of performance and 
standards for achieving stated criteria (Jonnson & 
Svingby, 2007). Holistic rubrics provide one score for 
the entire product, while analytic rubrics judge essential 
components separately (Arter & McTighe, 2000). For 
this study, an analytic rubric was developed so that each 
learning outcome could be scored. According to Banta 
et al. (2009), “Portfolio assessment of key outcomes 
can be graded using rubrics, yielding numerical scores 
that are reasonably reliable” (p. 11), if deliberate, 
considered effort is made to design and test the rubric. 

Specific to this project, two earlier faculty 
committee-generated documents were identified by the 
research team as key to anchoring rubric development. 
The PDP Learning Outcomes were expanded and then 
mapped to sections of “A Template for First-Year 
Seminars at IUPUI” (IUPUI, 2010), referencing the 
learning outcomes related to the Personal Development 
Plan. The descriptive characteristics of each rubric cell 
emerged as common themes were noted and the 
remaining outcomes and goals documented. Evaluative 
levels of achievement were guided by Bloom’s 
Taxonomy of Educational Objectives (see Gronlund & 
Brookhart, 2009) and Paul and Elder’s (2009) model of 
critical thinking. Specifically, evaluative levels were 
anchored around the constructs of knowledge, 
comprehension, application, and analysis. As a final 
step in the development of the rubric, the learning 
outcomes stated in each rubric cell were mapped to the 
current guided prompts provided in each section of the 
ePDP and then examined against the compatibility of 
the associated evaluative levels. 

The rubric was then piloted with three faculty 
members who had used the ePDP in their first-year 
seminar course for at least two semesters. The faculty 
members were asked to use the rubric to evaluate one 
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common ePDP and then a second ePDP of their 
choosing drawn from their own course. The group then 
provided feedback that informed the final iteration of 
the rubric used in this study. 

Data Collection and Analysis 

An email was sent to all faculty who used the 
ePDP in their first-year seminar course during the fall 
2012 semester inviting them to participate as raters in 
this study. Raters were provided with a gift card to the 
campus bookstore for their participation. Eleven faculty 
initially agreed to participate, with 10 ultimately 
following through on their commitment. The 10 faculty 
were each assigned nine or 10 ePDPs to review, so that 
each portfolio was scored by two reviewers. 

The most common type of reliability associated 
with the assessment of student work is inter-rater 
reliability. Inter-rater reliability is enhanced through a 
well-designed scoring rubric as well as by developing 
both consensus and consistency. Consensus refers to the 
degree to which raters provide the same score, while 
consistency provides a measure of correlation between 
the scores of the raters (Reddy & Andrade, 2010). Pilot 
testing of the rubric helped to ensure that the rubric was 
well-designed and provided initial feedback on levels of 
consensus. In order to enhance reliability, data 
collection occurred on the same day, with all reviewers 
in the same room. The session began by testing the 
rubric with two sample ePDPs. As raters compared and 
discussed scores, clarification was provided for 
wording within each cell, and scoring norms were 
agreed upon. 

After the initial introduction, which focused on 
developing inter-rater reliability, raters were asked to 
complete two tasks. First, raters provided a score for 
each competency on the rubric. Second, and most 
important for this study, raters highlighted words and 
phrases that supported their rubric score. Highlighting 
was done with colored markers so that student narrative 
could be associated with a specific learning outcome 
(e.g., all narrative that was evidence of self-awareness 
was highlighted in pink). The same highlighted 
narrative could be coded as applying toward more than 
one learning outcome. In addition, reviewers could 
highlight evidence as they saw fit. This resulted in 
differences in the identification of evidence; some 
reviewers highlighted full passages, others highlighted 
just phrases and words. 

Only text that was highlighted was included in the 
transcription for further coding and analysis. All words 
and phrases were transcribed in a separate document for 
each learning outcome. The transcriptions were 
uploaded into ATLAS.ti, a qualitative software analysis 
program. Transcripts were read, and an initial list of 
codes was developed deductively; additional codes 
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emerged inductively as the actual coding occurred. 
Codes were applied in the form of main categories and 
subcategories across all learning outcomes to allow for 
comparison of data across outcomes. Scores were 
totaled and compared between reviewers; in all but 
eight instances (16%), the ratings differed by less than 
one point per learning outcome and, from this, it was 
determined that there was consensus in ratings and an 
acceptable level of inter-rater reliability. 

Results 

Rubric scores on each of the five learning 
outcomes ranged from 0 (no evidence) to 4 (level of 
analysis; evidence that was exhibited at the level of 
analysis on Bloom’s taxonomy). The mean scores 
ranged from 1.68 to 1.16. Mean scores for each learning 
outcome are shown in Table 1. 

While the mean scores appear to be low when 
considered on the four-point rubric, because the 
rubric was based on levels of educational objectives 
and critical thinking, it is reasonable to expect that 
lower scores would be exhibited by students in their 
first-semester of college. For the purpose of this 
research, the types and content of responses 
uncovered through the analysis of student narrative 
within each learning outcome were as critical as the 
absolute score. 

Through coding, nine primary themes emerged 
across the array of learning outcomes. The number of 
phrases coded in each theme by learning outcome is 
shown in Table 2. Each phrase that was coded indicates 
a piece of authentic evidence identified by a faculty 
reviewer in support of the learning outcome. Table 1 
aligns closely with Table 2 in that the outcomes with 
the highest rubric scores had the highest number of 
pieces of authentic evidence. 

The largest portion of student narrative in the ePDP 
was associated with the learning outcomes of self- 
awareness and exploration of majors and careers. This 
finding is not surprising, given that new students are 
deeply engaged in decisions related to their purpose for 
enrolling in college, which is thought of most often in 
terms of majors and associated careers. In addition, 
because of the significant transition and newfound 
independence that first-year students experience, this 
year is also a time of reflection on one’s self as 
established views of the self are supported or 
challenged with each new situation a student 
encounters. 

What is perhaps most significant is that evidence to 
support the learning outcomes of self-awareness and 
exploration of major and career was found in narrative 
associated with other learning outcomes as well. This 
co-occurrence would appear to support the idea posited 
by Chen and Penny Light (2010) that “e-portfolios—as 
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both process and product—can promote deep learning 
and knowledge transfer by fostering the student’s 
ability to make connections between his or her learning 
experiences in a variety of classroom, workplace, and 
community settings” (p. 3). Knowledge transfer, in 
particular, appeared to be captured by reviewers when 
evidence they identified was coded as meeting more 
than one learning outcome. 

Self-Awareness 

Almost half of the coded evidence for self- 
awareness was in relation to students’ descriptions of 
their strengths, weaknesses, traits, and characteristics. 
This relationship between pieces of evidence is to be 
expected because the first section of the ePDP, titled 
About Me, asks students to describe themselves and their 
background as well as to discuss their personal strengths. 
Some students listed personality characteristics such as 
“slightly shy,” “adrenaline junky,” or “easy-going, 
energetic, friendly, and compassionate.” Others listed 
strengths such as “being a leader,” “hard-working,” 
“caring,” and “communication skills.” While most 
students provided a simple identification of strengths, 
one student expounded by providing very detailed 
examples of her strengths in action, how each strength 
was developed, and in what ways that strength will 
contribute to her future success. 

It was clear that at least a few sections of the 
first-year seminar led their students through 
structured activities to identify their strengths as 
students described their Holland career code, Myers- 
Briggs personality style, or results from the 
StrengthsQuest assessment tool in their description 
of themselves. 

As part of the discussion of themselves, students 
often noted the impact previous experiences had on 
their development. One student stated she had 
participated in many arts-related programs, “which I 
think has helped me so much on building my 
creativity skills.” Another, “worked around 15-20 
hours a week at a restaurant, which taught me a lot 
about work-ethic, taking pride in things I bought for 
myself, and effectively managing my time.” Other 
students listed sibling order, being raised in a rural 
community, their religious upbringing, or high 
school activities as being sources of the development 
of their characteristics and strengths. Participation in 
athletic teams was often mentioned. “I feel that 
baseball not only brought out the competitiveness in 
me but also strengthened my ability to lead” and 
“Being on Dance Team taught me how to jump into 
things and be spontaneous” are examples of student 
comments related to team participation. 

Some students were able to tie their 
characteristics and strengths to success in their chosen 
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Table 1 

Mean Scores for Each Learning Outcome 
Learning outcome M 

Self-Assessment and Awareness: Students identify success-related competencies 1.62 

Exploration of Major and Career: Students research and identify realistic and ^ 

informed academic and career goals 

Goal Setting: students indicate short and longer term goals as well as connect ^ ^ 

personal values and life purpose to the motivation behind their goals 


Planning: students locate programs, information, people, and opportunities to support 
and reality test their goals 

Evaluation: Students analyze their academic program in terms of progress toward 
academic and career goals 


Table 2 


Coded Phrases in Each Theme by Learning Outcome 


Coded phrases 

Self- 

assessment 

and 

awareness 

Exploration 
of major and 
career 

Goal setting 

Planning 

Evaluation 

Total 

Understanding of Self 

1031 

303 

58 

98 

314 

1804 

Major and Career 

83 

646 

171 

151 

74 

1125 

Values and Purpose 

177 

91 

69 

39 

30 

406 

Personal Development 

64 

34 

57 

120 

83 

358 

Grades 

10 

50 

50 

104 

21 

235 

College Transition 

2 

1 

5 

5 

105 

118 

High Impact Practice 

3 

6 

32 

65 

5 

111 

Campus Involvement 

2 

0 

18 

38 

13 

71 

Giving Back to Others 

12 

8 

16 

26 

7 

69 

Total 

1384 

1139 

476 

646 

652 

4297 


major and career. “My strengths are my people skills 
and my persuasive skills, which I believe will help me 
in being a lawyer” and “I want to each English so that 
I can share my love of reading and writing with 
others” illustrate of this type of linked thinking. 
Another said, 

Anyone who has known me since birth has 
described me as very happy. They would say I am 
very compassionate. This is important to me 
because my dream is to work in an Emergency 
room, and feeling for the patients and relating to 
them will be very important. It is important to 
know the technical side of nursing but also the 
human side and be able to sympathize with the 
patient. 

One more example of linking personal characteristics to 
major and career selection is, “Being a quiet and 
organized person will help me in the career in 
philanthropy because it is not always about being the 
center of attention, it’s about being respectful, 
organized, and hard-working—all the things I do well.” 


Finally, students offered narrative that suggested 
they want to further develop their skills, knowledge, 
and characteristics through college experiences. One 
student stated, “If I could get myself involved in both of 
these areas, then I would form good communication 
skills, teamwork, and friendships with lots of people.” 
Other students said, “improve my communication 
skills,” “improve my time management skills,” and less 
specifically, “over time develop more skills that will 
help me be the best I can be.” 

Exploration of Major and Career 

The learning outcome exploration of major and 
career had the second highest number of pieces of 
authentic evidence cited by reviewers. The majority of 
students stated a specific major or career goal. Other 
students indicated an area of study that interested them 
such as the medical field or “working with charities.” 

The focus on exploring majors and careers 
appeared most often in narrative related to career 
research. Students spoke of job shadowing and 
internship experiences that guided or confirmed their 
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choice of career. For example, a dental hygiene student 
said, 

I have excellent verbal skills, while I did my 
externship at Pritchett orthodontics, I was 
responsible for seating patients and making sure 
they were comfortable while their braces were 
being placed on, or tightened. My eyesight is 
perfect, and I do fantastic with hands on. For 
example, my externship included me placing bands 
on patient’s braces. 

Another student stated, 

After going to the cancer center ... I definitely 
think Fd be interested in respiratory therapy. I 
really liked the relationships and achievement 
aspect of the job. The journey you take with each 
patient seems so special and to see them overcome 
the fight and you helped them do it seems so 
special. 

In addition, students indicated level of education, salary 
ranges, and occupational outlook as evidence of their 
career research. “To be a teacher, you need to be patient, 
caring, and understanding. As a teacher, you would be 
teaching children new skills and preparing lesson plans. 
As [sic] teacher usually needs a Bachelors degree. 
According to the Bureau of Labor Statistics, an elementary 
education teacher makes about $51,000 per year.” 

Many students wrote about the knowledge, skills, 
and interests a person in their chosen career must have. 
“Some personal characteristics of someone working in 
law enforcement would be someone who is a leader, 
confident, social, flexible, and strong willed.” 
“Working with deadlines,” “communicate with others,” 
“empathetic,” “good manual dexterity,” and "honesty 
and humor” were given as examples of characteristics 
needed for success in their chosen career. 

There was also evidence of students identifying 
characteristics, skills, and strengths they possessed and 
their relationships to chosen majors and careers. “1 truly 
do have a passion for helping people” and “always had 
an interest in working with kids” are two such 
examples. One student wrote, “Philanthropic Studies 
will allow me to work closely with charities and really 
allow me to make a difference in the world.” Most 
statements provided by students were stated in very 
general terms about wanting to help others, work with 
children, and make people happy. 

Largely, the authentic evidence that spoke to 
majors and career selection was information-based and 
showed some links to self-understanding. As can be 
seen in the reviewers’ low rubric scores, however, the 
evidence was not presented with a high level of critical 
thinking or depth of analysis. 
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Goal Setting 

The majority of student narrative serving as 
evidence of goal setting was around the notion of 
majors and careers. Most students indicated a specific 
major or career objective and stated these goals. 
Student narrative included statements such as, “Since 
grade school, I have known that I wanted to pursue 
medical school”; “When I graduate, 1 would like to 
become a Child Life Specialist”; and “1 hope to one 
day be promoted to a charge nurse.” Students also 
stated goals in terms of their intended major. Others 
focused on more immediate goals, such as completing 
pre-requisite courses, changing their major, and 
gaining admission to a competitive academic program. 
Many students set specific grade-point average goals, 
particularly those students who have minimum grade 
requirements for admission to their intended major. 
Very few students articulated the connection between 
their intended major and career as well as this student, 
“college education with this major . . . will open many 
door [sic] to my professional goal, through the 
rigorous analytical chemistry specialization offered at 
the School of Science.” Overall, students were able to 
state their major and career goals, but did not show 
clear evidence of being able to tie the major goal to 
career aspirations. 

Some students, however, were able to identify the 
values and purpose that supported their choice of major 
or career. One student indicated that s/he wanted to 
“pursue a life of helping students learn.” Statements of 
values and purpose also related to their reasons for 
enrolling in college. Statements such as, “I am very 
driven and passionate about my college education 
because without that life is going to be pretty tough”; 
“My goal is to create the foundation that helps me with 
my drive to get good grades and be very successful in 
the workplace”; and “I also want to gain the knowledge 
that I am going to need to know to make it in the real 
world” indicate that students perceive the value of 
college to be related to a stable economic future. 

Students also articulated goals related to their 
personal development while in college. Some students 
spoke of developing skills such as time management, 
communication, and independence. Others spoke more 
broadly about their vision for their future self. “1 hope 
that as a person, college will develop me into a more 
outgoing, confident human being.” A common thread 
was for students to speak of their desire to understand 
different cultures. Students spoke both broadly and 
specifically about their desire to be exposed to diverse 
peoples and perspectives. A general statement was 
made by one student: “As a citizen, I hope to gain 
experiences with others from different backgrounds and 
walks of life, so I can better learn what it means to live 
and function in the society we live in.” Others spoke 
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specifically, “I plan to travel abroad to the Dominican 
Republic" and 

Even for just a couple of weeks I want to study in 
Japan. The culture has always been a huge interest 
to me and I would love to be involved over there 
with the program as it would really give me so 
much more knowledge then [sic] just the culture 
and volunteering here. 

Overall, there was authentic evidence that students 
completing the ePDP were able to articulate their goals 
as related to major, career, and personal development. 
However, the degree to which students were able to 
state goals clearly and relate them to personal values 
was mixed. Very little evidence was found of students 
being able to weave a coherent narrative that articulated 
goals, the underlying values and purpose guiding the 
goal, and campus opportunities that would support 
achievement of the stated goals. This finding was 
shown both through the authentic evidence as well as 
the low reviewer rubric scores. 

Planning 

Evidence of student planning fell into three 
categories: course plans and schedules, high-impact 
practices, and campus resources to support achievement 
of goals. All three were presented by students as steps 
they would take to achieve their major and career goals. 
Course planning is a required component of the ePDP 
with a link directly to the student record system’s 
academic planner and degree audit software. All 
students are required to complete a two to four-year 
course plan. Some students provided additional lists of 
courses that they planned to take in order to confirm 
their choice of major. 

Students also identified high-impact practices in 
which they plan to participate in order to support their 
learning, career goals, and personal development. It is 
important to note that IUPUI encourages all students to 
complete at least two RISE (research, international, 
service learning, and experiential) opportunities before 
they graduate. In addition to the four curricular and co- 
curricular opportunities in the RISE program at IUPUI, 
students noted plans to get involved on campus by 
joining student organizations or finding on-campus 
student employment positions that would support their 
networking and allow them to gain experiences related 
to their major. 

Study abroad was the most commonly cited high- 
impact practice that students incorporated into their 
planning. Some students spoke of international study as 
a way to develop career-related skills in statements such 
as, "there are different types of diseases in other parts 
of the world that are not common in the US so I could 
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benefit by learning about the other sicknesses around 
the world." Others sought to participate in international 
experiences for personal development: “Study abroad 
will help me become a better well-rounded person by 
learning other cultures.” Others indicated specific 
locales they wanted to visit due to personal interest or 
family heritage. 

Internships were often noted as part of students’ 
college plans. Unlike the student narratives on study 
abroad, which often included a short statement as to 
why the student wanted to engage in international 
travel, students rarely indicated the benefit of an 
internship or what they would learn. Most students 
simply listed getting an internship as a way to gain 
experience. Two students noted the networking that 
accompanies internship opportunities. One student 
stated, “I have heard that both internship programs are 
good, plus there are graduating students and professors 
that have ties to both of them, which would make it 
easy for me to get one.” Another indicated, “By 
participating in an internship, it could help you get 
inside connections and potentially allow you to get a 
job easier.” 

Students cited a wide range of people and campus 
resources that they planned to utilize to enhance their 
college and career success. Friends, campus mentors, 
academic advisors, and faculty members were often 
cited as individuals who could provide both information 
and support. Campus resources including the Math 
Assistance Center, academic mentoring sessions, and 
the library were referenced as academic supports that 
would support earning good grades. Overall, student 
statements about people and resources were presented 
at the lowest level of Bloom’s taxonomy through 
statements or lists of resources; few artifacts contained 
more in-depth information on how the resource would 
specifically assist the student. This perhaps could be 
attributable to the fact that first-year students had not 
yet acquired much direct experience in using these 
resources, though they clearly were aware that they 
existed and could support their college experience. 

Evaluation 

The evaluation outcome had the lowest rubric 
scores. Thus, one would expect to see the lowest 
number of artifacts. This expectation, however, did not 
hold true, suggesting that students did indeed show 
beginning evidence of evaluation, although their 
narratives reflected the lowest levels of Bloom’s 
taxonomy and relatively weak critical thinking. Most 
evidence of evaluation was related to the transition to 
college and understanding of self. This type of narrative 
is to be expected in that the first-semester of college is 
one in which students face great transition and, in doing 
so, are continuously measuring their current skills. 
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abilities, knowledge, and identity against new 
challenges and opportunities. 

Most often, students spoke of the transition to 
college in terms of challenges faced and of what was 
learned. Students frequently used the phrase “culture 
shock’’ to describe the transition to college. The culture 
shock referred to the size of the campus, being in an 
urban environment, and not being with friends and 
family. Some spoke of independence and responsibility: 

When I came to college there was [sic] a few 
things I had to adapt to right away. For example I 
didn’t have my parents to depend on anymore. I 
think this was the biggest shock for me. It was the 
first time I have had to fully depend on myself. 

Students named making doctor’s appointments, opening 
bank accounts, and using an insurance card as new 
challenges. One student said, 

It has been a struggle to figure out how to function 
on my own as an adult. I was never used to 
creating my own budgets, doing all of my own 
shopping, cooking, and cleaning in an apartment by 
myself. I’ve struggle [sic] with balancing these 
things with school. 

Other students noted the differences in college- 
level learning. One student stated. 

Going off what I said before about college being 
much harder than high school, I think that it is just 
a whole other level of learning. It is more self¬ 
learning based . . . You are just expected to figure 
out more on your own. 

Another agreed by stating, “College learning puts more 
responsibility in our hands.” 

Narrative did show students identifying things that 
assisted them with their transition to college: “I took 
advantage of spring and fall preview days” and “I also 
came [and] visited the school multiple times.” “One 
thing I used to help my transition in IUPUI was 
participating in my learning community.” Friends who 
had previously attended IUPUI were also named as 
assisting with the shift to the new environment. 

Within the evaluation learning outcome, student 
narrative indicating self-understanding fell primarily 
into two categories: understanding of self in relation to 
others and college success skills. Students spoke about 
participation in activities such as visiting an area prison, 
volunteering at a homeless shelter, and touring facilities 
related to future career goals. One student stated, “[The 
activity] made me want to get way more involved with 
my community and maybe even outside of my 
community.” Another wrote about meeting “The kind 
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of people I want to work with for the rest of my life and 
I was more thankful for being there than they were for 
me being there.” Another related the experience directly 
to career goals: “I can definitely relate this to my future 
career because these are the kinds of things I want to do 
and the type of people I want to work with once I obtain 
my law degree.” 

The transition to college, as related to evaluation of 
progress toward academic and career goals, was seen in 
artifacts focusing primarily on time management and 
the heavy load of studying. Perhaps one student said it 
best: 

The differences in time management have been 
something new to college. Instead of going to 
school at 8 am every morning, I have to remember 
that I go to school at 9 am on two days and 3 pm 
on other days. This means that I have to set up 
different schedules for different days, which is not 
something I’m used to. I had trouble with 
maintaining a regular sleeping schedule, and was 
tired all the time. However, I have learned to go to 
bed at the same time each night and to wake up at 
the same time each morning. This kept my sleeping 
schedule regular, and meant that I alternated when 
I did other things, rather than when I slept. This 
was one of the most important lessons I learned 
this semester. Keeping on top of my schedule will 
help ensure that I can succeed in later semesters. 

Other students spoke more generally about struggles 
with time and workload: “I think that the biggest 
challenge I faced was managing the time from 
homework and classes to spending time with my 
friends”; “Challenge in terms of finding success was 
time-management”; and “One of the biggest challenges 
I have faced this semester is time management and 
getting my priorities in line.” As in the previous 
section, one student was able to relate the insights from 
evaluation to career goals: 

Throughout the semester I have used his tips, 
advice, and even his silly games to take the stress 
off me. This made me realize that I can actually be 
good at school and that I could possibly uses [sic] 
these techniques with the children I want to help as 
a psychologist when I am older. 

Several artifacts indicated evaluative insights 
gained from the first semester at college. One student 
stated, “This experience is far off from what I was told 
in high school.” Another spoke more specifically by 
stating, “College has increased my maturity, my work 
ethic, and my determination.” 

In summary, using an analytic rubric designed to 
evaluate student narrative in an ePortfolio, reviewers 
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found low-level evidence across all five learning 
outcomes for the first-year seminar course. Authentic 
evidence supported the low-level ratings in that student 
narrative was presented at the initial levels of Bloom’s 
taxonomy—primarily identification and description. It 
is important to note that in most cases, students do not 
revise and resubmit work presented within the ePDP 
during the first-year seminar, as it is assumed revision 
will occur when the students update their portfolio 
throughout their college career. Therefore, the low-level 
ratings are a measure of learning at a specific point in 
time and after one occurrence of responding to the 
reflective prompts. 

As mentioned earlier, one of benefits of using 
ePortfolios is the transfer of knowledge across 
concepts. There was some evidence of this transfer of 
knowledge in the fact that faculty coded student phrases 
as aligning with multiple learning outcomes, 
particularly across the constructs of self-awareness and 
exploration of major and career. However, often 
students did not articulate clearly connections between 
learning across outcomes suggesting, again, that 
students were in the initial stages of this connected 
thinking. It is possible that the appearance of transfer of 
learning may have been facilitated by the order in 
which the seven distinct sections of the ePDP are 
implemented. For example, most faculty assign the 
foundational About Me section first, followed by 
Educational Goals and Plans and Career Goals. It is 
reasonable to assume that students might utilize the 
narrative provided in the About Me section to support 
their educational choices and career goals. 

Discussion 

Implications for the Use of the ePDP in First-Year 
Seminars 

The findings of this study reveal several 
implications for the use of an ePortfolio in a first-year 
seminar course. First, in terms of the research 
methodology, it became clear through the coding 
process that while there was an acceptable level of 
inter-rater reliability on the individual rubric cells, there 
frequently were differing interpretations of the overall 
learning outcome. These differing interpretations were 
revealed when reviewers had similar rubric scores but 
used evidence in very different ways. For example, a 
student comment about wanting to help people could 
have been coded as an underlying value guiding choice 
of major and career by one reviewer (therefore 
supporting the exploration of major and career learning 
outcome), or as evidence of understanding of self (self- 
awareness learning outcome) by another, or even 
possibly both by yet another reviewer. Consistency in 
rubric scores suggests that the rubric was a reliable tool 
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for the study, but that clearer definitions and agreement 
on the learning outcomes needs to be developed. 

In terms of implications for practice, a 
determination needs to be made about the level of 
outcome desired from first-semester students enrolled 
in the course. This study revealed a preponderance of 
rubric scores that aligned with the most basic levels of 
Bloom’s taxonomy. These low level scores could be 
considered appropriate given that the students are in 
their first-semester of college. However, with the use of 
appropriate pedagogical strategies, it is possible for 
first-year students to demonstrate higher levels of 
learning. Rubric scores provided an objective measure 
of learning; whether or not the objective measure 
matches the desired learning outcome must be 
determined by faculty. Because the ePDP is designed 
for use across students’ four years of enrollment, lower 
levels of proficiency for learning outcomes have, to 
date, been deemed acceptable for the first-year seminar 
course. It is assumed that evidence of higher levels of 
achievement on Bloom’s taxonomy and aspects of 
critical thinking will be found as a student continues to 
revise the ePDP as they progress through college. 

If, however, it is determined that a higher level of 
achievement is desired from the first-year seminar (on 
all or some of the learning outcomes), the scaffolding 
and guidance associated with the ePDP will need to be 
enhanced. For example, if greater connections between 
individual strengths, choice of major, and career 
selection are sought, students will need to be led 
through a series of classroom activities designed to help 
move students from identification and descriptions of 
strengths, majors and careers to provision of examples 
and statements of integration. Higher levels of 
achievement may also require more classroom focus on 
scaffolding critical thinking and critically reflective 
writing (Ash & Clayton, 2009). 

Greater focus on the pedagogy associated with the 
use of an ePortfolio in the first-year seminar also has 
implications for faculty development. In fact, in the 
model for ePortfolio use developed by the Making 
Connections National Resource Center (2013) Connect 
to Learning project indicates that faculty development 
is a major component of ePortfolio implementation. As 
related to this study, faculty development around the 
meaning of stated learning outcomes needs to occur. 
The learning outcomes are part of a common document 
provided to all first-year seminar instructors (IUPUI, 
2010) but because there are over 100 sections of the 
course offered each fall, the learning outcomes become 
subject to the individual interpretations of instructors. 
Further, faculty involved in this study had differing 
levels of experience using rubrics; some had used a 
rubric to grade the ePDP while others had not. Faculty 
involved in this study indicated that using this rubric 
helped them think through not only the outcomes and 
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implementation of the ePDP in their course, but also 
their methodology for grading. Faculty development on 
both grading student narrative and assessment of 
learning outcomes using a rubric is an important 
component of using an ePortfolio in a course. 

Limitations 

Two primary limitations must be considered when 
interpreting the results of this study. First, because of 
limited access to ePDPs in the portfolio platform, a 
convenient sample was used. Consequently, and 
keeping in mind that qualitative research is not meant to 
be generalized, the portfolios reviewed may not 
constitute a representative sample of all ePDPs 
submitted in the fall 2012 semester. Second, this study 
did not consider the implementation or pedagogical 
methods associated with ePDP. As mentioned earlier, 
each faculty member is able to implement the ePDP in 
their course as he or she sees fit. Therefore, it is likely 
that the scaffolding and guidance for reflection varied 
across first-year seminar sections which, in turn, may 
have affected the depth and focus of students’ narrative. 

Suggestions for Future Research 

The results and limitations of this study lead to 
additional questions that could enhance the 
understanding and use of student narrative found in 
ePortfolios as a source of authentic assessment. Co¬ 
occurrence of evidence in this study became apparent 
through the numerical summary of data across learning 
outcomes. Because this finding arose inductively from 
the data, student narrative was not coded with co¬ 
occurrence in mind. Research with coding structures 
that clearly identify instances of one data point being 
used to support multiple student learning outcomes— 
that is to say, the ability of students to connect their 
learning and thinking across conceptual lines—would 
contribute to the literature on folio thinking (Chen & 
Mazow, 2002, as cited in Chen & Penny Light, 2012) 
Further, while inklings of the ideal of folio thinking 
were found in some aspects of the analysis, to truly test 
the power of ePortfolios as tools for students to engage 
in the transfer of knowledge and weave a consistent and 
coherent story of themselves, their college experience, 
and their goals, the ePDPs should be studied 
individually with a rubric that focuses on using aspects 
of critical thinking to assess the ability of the student to 
make connections across content and sections of the 
ePDP. 

Conclusion 

This study found that student narrative from an 
ePortfolio can be used as a reliable form of evidence for 
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authentic assessment to measure learning outcomes 
associated with a first-year seminar course. Findings 
indicated that students achieved the learning outcomes 
at the identification and description levels of Bloom’s 
taxonomy and provided authentic evidence supporting 
these scores. The student narrative identified as 
evidence generally lacked the depth, analysis, and 
connections found at high levels of learning. 

The results from this study have important 
implications for literatures related to ePortfolios, first- 
year seminars, assessment and, perhaps most important, 
the intersection of all three as a way to maximize the 
efficacy of high-impact practices and assess the 
outcomes of such interventions. The authentic evidence 
uncovered through the ePDPs supported Banta’s (2003) 
assertion that portfolios can provide insight into what 
students are learning ,as well as how they are learning, 
as both content and levels of learning were uncovered. 
It is posited that the degree to which student learning 
outcomes were met (or not met) is influenced by the 
classroom activities and guidance surrounding the use 
of the ePDP as a pedagogical tool, suggesting that 
learning in first-year seminars can be impacted by the 
inclusion of an ePortfolio. The use of student narrative 
as a source of evidence about learning outcomes, while 
time-intensive, provided depth of understanding related 
to student achievement that is not available through 
more traditional course evaluation methods. The 
transformation of higher education to enhance student 
success and learning can be maximized when powerful 
practices are brought together. 
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